Introduction¶

This assignment aims to compare the performance of various image smoothing filters applied to noisy images. The filters under consideration include simple smoothing filters such as Box filter, Gaussian filter, and Median filter, as well as advanced filters like Bilateral filter and Adaptive Median filter. The evaluation will focus on noise removal effectiveness, edge preservation, computational efficiency, and the influence of kernel size on the performance of each filter. Quantitative metrics such as Mean Squared Error (MSE) and Peak Signal-to-Noise Ratio (PSNR) will be used for comparison.

Filters to be Used:¶

1. Box Filter:

   • Averages the pixel values within a defined kernel size, effective for noise reduction but can blur edges.

2. Median Filter:

   • Replaces each pixel with the median value of its neighboring pixels, effective for removing salt-and-pepper noise while preserving edges.

3. Gaussian Filter:

   • Applies a Gaussian function to the neighboring pixels, smoothing the image while preserving edges better than the Box filter.

4. Bilateral Filter:

   • Considers both spatial distance and intensity difference between pixels, smoothing the image while preserving edges by combining Gaussian smoothing in both spatial and intensity domains.

5. Adaptive Median Filter:

   • Adapts the size of the filtering window based on local image characteristics, effectively removing noise while preserving edges and fine details.

6. Adaptive Mean Filter:

   • The Adaptive Mean Filter reduces noise by adjusting the filtering process based on local variance. It calculates the local mean and variance within a kernel and adapts accordingly, effectively preserving edges and fine details while reducing noise.

The comparison of these filters will provide insights into their strengths and weaknesses in different aspects of image processing, highlighting their suitability for various applications.

Description of Images¶

The images used in this analysis are categorized based on the level of detail they contain. The categories are as follows:

1. Low Details:

   • These images contain minimal detail and are typically characterized by large, uniform regions with little texture or fine structure. They are often used to evaluate the performance of filters in preserving smooth areas while removing noise.

2. Medium Details:

   • These images contain a moderate amount of detail, including some texture and fine structures. They provide a balanced scenario to test the effectiveness of filters in both noise removal and edge preservation.

3. High Details:

   • These images are rich in detail, with intricate textures and fine structures. They are used to assess the ability of filters to preserve edges and fine details while effectively reducing noise.

The images are processed and analyzed to compare the performance of various filters, including Box filter, Gaussian filter, Median filter, Bilateral filter, Adaptive Median filter, and Canny edge detector. The evaluation focuses on noise removal effectiveness, edge preservation, computational efficiency, and the influence of kernel size on the performance of each filter.

The cell below shows the images to be worked on for this assignment.

Experiments¶

Noise Generation and Application on Images¶

In this section, we will discuss the process of generating noise and applying it to images. Noise is an unwanted random variation in the pixel values of an image, which can degrade the visual quality and affect the performance of image processing algorithms. Different types of noise can be introduced to simulate real-world scenarios and evaluate the robustness of image processing techniques.

Types of Noise:¶

1. Gaussian Noise:

   • Gaussian noise is characterized by a normal distribution of pixel values around the mean. It is commonly used to simulate sensor noise in digital images. The noise level can be controlled by adjusting the mean and standard deviation of the distribution.

2. Salt and Pepper Noise:

   • Salt and pepper noise, also known as impulse noise, is characterized by randomly occurring white and black pixels in the image. It is typically caused by faulty camera sensors, transmission errors, or corrupted image files. The noise level can be controlled by adjusting the probability of occurrence of the white and black pixels.

Applying Noise¶

To evaluate the performance of various image smoothing filters, we apply different types of noise to the original images. This allows us to test the effectiveness of the filters in removing noise while preserving important image details such as edges and textures.

The following steps are involved in applying noise to images:

1. Load the Original Image:

   • The original image is loaded from the specified directory.

2. Generate Noise:

   • Noise is generated based on the specified type (Gaussian or Salt and Pepper) and noise level (low, medium, high).

Applying Noise to Images:¶

• The cell below calls a function that creates a pandas dataframe for each image. This dataframe indices has the noise variations of the image, including the original image.

• For instance, df['Gaussian Noise (medium)'] fetches the image corrupted with medium gaussian noise.

Visualization of different noises on the 3 images:¶

The cell below shows each of noises with different levels of intensities, applied on all of the images.

#¶

Filter Application¶

The next cell applies each filter to each image and saves the results in a tree-structured directory. This allows for organized storage and easy retrieval of the filtered images for further analysis.

Visualizing Noisy vs Filtered Image¶

In the following cells, we will visualize the difference between a noisy image and its filtered version using a kernel size of 5. The noise applied is gaussian for the first, salt and pepper for the next. This process involves applying a filter to the noisy image to reduce noise and enhance the quality of the image.

Steps:¶

1. Load the Noisy Image: We start by loading the noisy image that we want to filter.
2. Apply the Filter: We use a filtering function with a kernel size of 5 to process the noisy image. The kernel size determines the area of the image that the filter will consider when smoothing the image.
3. Visualize the Results: Finally, we display both the original noisy image and the filtered image side by side for comparison.

Code Explanation:¶

• Function Call: The function visualize_noisy_vs_filtered(noisy_image, kernel_size=5) is called, where noisy_image is the input image and kernel_size=5 specifies the size of the filter kernel.
• Visualization: The function handles the visualization, showing the noisy image on the left and the filtered image on the right.

This comparison helps in understanding the effectiveness of the filtering process in reducing noise and improving image quality.

Observation on Gaussian Noise¶

By examining the images with Gaussian noise applied, it is observed that the Bilateral Filter performed the best in terms of noise reduction. The Bilateral Filter effectively smooths the image while maintaining a good balance between noise removal and detail preservation. The Gaussian Filter also performed well, providing a smooth result with minimal noise, but it slightly blurred the image compared to the Bilateral Filter. The Median Filter was effective in reducing noise but introduced some artifacts in the image. The Box Filter was the least effective, as it resulted in significant blurring and loss of detail.

Observation on Salt and Pepper Noise¶

By examining the images with Salt and Pepper noise applied, it is observed that the Median Filter performed exceptionally well, eliminating all the noise while preserving the edges and details of the image. The Adaptive Median Filter also performed well, eliminating most of the noise and maintaining a good balance between noise removal and detail preservation. The Bilateral Filter and Gaussian Filter were less effective in removing Salt and Pepper noise, as they left some noise in the image. The Box Filter was the least effective, resulting in significant blurring and loss of detail while failing to remove all the noise.

Results and Discussion¶

Metrics: MSE and PSNR¶

In this section, we evaluate the performance of various filters using two quantitative metrics: Mean Squared Error (MSE) and Peak Signal-to-Noise Ratio (PSNR). These metrics help in assessing the effectiveness of the filters in noise removal and detail preservation.

Mean Squared Error (MSE)¶

MSE measures the average squared difference between the original and filtered images. A lower MSE value indicates better performance in terms of noise reduction and detail preservation.

Peak Signal-to-Noise Ratio (PSNR)¶

PSNR is a ratio between the maximum possible power of a signal and the power of corrupting noise. It is expressed in decibels (dB). A higher PSNR value indicates better performance in terms of noise reduction and detail preservation.

Visualization and Comparison¶

The MSE and PSNR values for each filter were calculated and visualized using line charts. These charts compare the performance of different filters across various noise levels and kernel sizes.

• MSE vs. Kernel Size: This chart shows how the MSE values change with different kernel sizes for each filter and noise type.
• PSNR vs. Kernel Size: This chart shows how the PSNR values change with different kernel sizes for each filter and noise type.

These visualizations provide insights into the effectiveness of each filter in different scenarios, helping to identify the best filter for specific noise types and levels.

Filter Performance on Different Noises using MSE and PSNR¶

1. Box Filter¶

• Gaussian Noise:
  • MSE: Moderate, as the Box filter averages pixel values, reducing noise but blurring edges.
  • PSNR: Moderate, indicating some noise reduction but loss of detail.
• Salt and Pepper Noise:
  • MSE: High, as the Box filter is less effective at removing impulse noise.
  • PSNR: Low, indicating poor performance in noise removal and edge preservation.

2. Median Filter¶

• Gaussian Noise:
  • MSE: Moderate, effective in reducing noise but may not preserve all details.
  • PSNR: Moderate to high, indicating good noise reduction with some loss of detail.
• Salt and Pepper Noise:
  • MSE: Low, highly effective at removing impulse noise.
  • PSNR: High, indicating excellent noise removal and edge preservation.

3. Gaussian Filter¶

• Gaussian Noise:
  • MSE: Low, as the Gaussian filter is well-suited for reducing Gaussian noise.
  • PSNR: High, indicating effective noise reduction while preserving edges.
• Salt and Pepper Noise:
  • MSE: High, as the Gaussian filter is less effective at removing impulse noise.
  • PSNR: Low, indicating poor performance in noise removal and edge preservation.

4. Bilateral Filter¶

• Gaussian Noise:
  • MSE: Low, effectively reduces noise while preserving edges.
  • PSNR: High, indicating superior performance in noise reduction and edge preservation.
• Salt and Pepper Noise:
  • MSE: Moderate, better than Gaussian filter but not as effective as Median filter.
  • PSNR: Moderate to high, indicating good noise reduction with some preservation of edges.

5. Adaptive Median Filter¶

• Gaussian Noise:
  • MSE: Low, adapts well to varying noise levels.
  • PSNR: High, indicating excellent noise removal and edge preservation.
• Salt and Pepper Noise:
  • MSE: Low, highly effective at removing impulse noise.
  • PSNR: High, indicating superior performance in noise removal and edge preservation.

6. Adaptive Mean Filter¶

• Gaussian Noise:
  • MSE: Low, adjusts based on local variance, effectively reducing noise.
  • PSNR: High, indicating good balance between noise reduction and edge preservation.
• Salt and Pepper Noise:
  • MSE: Moderate, better than Gaussian filter but not as effective as Median filter.
  • PSNR: Moderate to high, indicating good noise reduction with some preservation of edges.

7. Effect of Kernel Size on Denoising Performance¶

The kernel size is a critical parameter in image denoising techniques. It determines the size of the neighborhood around each pixel that is considered during the filtering process. The choice of kernel size can significantly impact the effectiveness of noise removal. Below, we discuss the effects of different kernel sizes on denoising performance based on the observed Mean Squared Error (MSE) and Peak Signal-to-Noise Ratio (PSNR):

1. Small Kernel Size¶

• Noise Removal: A small kernel size may not be sufficient to remove significant noise, especially in highly noisy images. It may only smooth out minor variations and leave some noise intact.
• MSE: Lower MSE, as the filter does not overly smooth the image, preserving more details.
• PSNR: Higher PSNR, indicating better preservation of the original image quality.

2. Medium Kernel Size¶

• Noise Removal: A medium kernel size provides a balance between noise removal and edge preservation. It can effectively reduce moderate noise while maintaining important image features.
• MSE: Moderate MSE, as the filter balances noise reduction and detail preservation.
• PSNR: Moderate PSNR, indicating a good trade-off between noise removal and image quality.

3. Large Kernel Size¶

• Noise Removal: Large kernel sizes are more effective at removing significant noise, as they consider a larger neighborhood around each pixel. However, this can lead to over-smoothing.
• MSE: Higher MSE, as the filter tends to blur edges and fine details, leading to a loss of important image features.
• PSNR: Lower PSNR, indicating worse preservation of the original image quality due to excessive smoothing.

Summary¶

• Best Performance on Gaussian Noise: Gaussian, Bilateral, Adaptive Median, and Adaptive Mean filters. The median filter showed superior performance in the low detailed image.

• Best Performance on Salt and Pepper Noise: Median, Adaptive Median, and Bilateral filters.

• Moderate Performance: Gaussian filter on Salt and Pepper noise, Adaptive Mean filter on Salt and Pepper noise.

• Lower Performance: Box filter on both Gaussian and Salt and Pepper noise.

• Small Kernels: Best for preserving edges and fine details, resulting in lower MSE and higher PSNR.

• Medium Kernels: Provide a good balance between noise removal and edge preservation, with moderate MSE and PSNR.

• Large Kernels: Effective at removing significant noise but tend to blur edges and fine details, resulting in higher MSE and lower PSNR.

Filters Computational efficiency¶

Computational efficiency is a critical aspect when evaluating the performance of image smoothing filters. It refers to the amount of computational resources, such as time and memory, required to apply a filter to an image. Efficient filters can process images quickly and with minimal resource usage, making them suitable for real-time applications and large-scale image processing tasks.

In this section, we analyze the computational efficiency of various filters by measuring the time taken to apply each filter to images with different kernel sizes. The results are visualized to compare the performance of filters like Box filter, Gaussian filter, Median filter, Bilateral filter, and Adaptive Median filter. This analysis helps in understanding the trade-offs between noise removal effectiveness and computational cost, guiding the selection of appropriate filters for specific applications.

The cell below calls the function collect_filter_times, which returns a dataframe containing timing information for each of the filters, then prints the dataframe.

Computational Efficiency¶

• Box Filter: Fastest, with minimal computational cost, as it involves only simple averaging operations.

• Gaussian Filter: Very efficient, with low computational cost. Computational burden is similar to that of the box filter, the only difference is the weights assigned to the kernel.

• Median Filter: Moderate computational cost, increases with kernel size. This is due to it being a non-linear operator. As the kernel size increases, the number of elements that need to be sorted increase, which results in a time increase.

• Bilateral Filter: Higher computational cost, increases significantly with kernel size.

• Adaptive Median Filter: High computational cost, increases with kernel size.

• Adaptive Mean Filter: High computational cost, varies with local variance and kernel size.

Edge preservation test:¶

This section of the code is responsible for testing the edge preservation capabilities of the image processing algorithm. It utilizes the Canny edge detection method to identify and highlight the edges within an image. Canny edge detection is a multi-stage algorithm that involves noise reduction, gradient calculation, non-maximum suppression, and edge tracking by hysteresis to produce a binary image showing the detected edges. The purpose of this test is to ensure that the edges in the processed image are preserved accurately and distinctly.

Edge Preservation Discussion¶

Edge preservation is a crucial aspect of image filtering, as it ensures that important structural details in the image are maintained while reducing noise. In this section, we discuss the edge preservation capabilities of various filters applied to images with different types of noise and kernel sizes.

Effect of Increasing Kernel Size on Edges¶

1. Small Kernel Size (3x3, 5x5):

   • Noise Removal: Small kernel sizes are effective in preserving edges while performing moderate noise removal. They consider a smaller neighborhood, which helps in maintaining the sharpness of edges.
   • Edge Preservation: High, as the filter does not overly smooth the image, preserving more details and edges.
   • Filters: Median Filter, Bilateral Filter, Adaptive Median Filter.

2. Medium Kernel Size (7x7, 9x9):

   • Noise Removal: Medium kernel sizes provide a balance between noise removal and edge preservation. They are effective in reducing moderate noise while maintaining important image features.
   • Edge Preservation: Moderate, as the filter balances noise reduction and detail preservation.
   • Filters: Gaussian Filter, Bilateral Filter, Adaptive Mean Filter.

3. Large Kernel Size (11x11, 13x13, 15x15, 17x17, 19x19):

   • Noise Removal: Large kernel sizes are more effective at removing significant noise, as they consider a larger neighborhood around each pixel. However, this can lead to over-smoothing.

   • Edge Preservation: Low, as the filter tends to blur edges and fine details, leading to a loss of important image features.

   • Filters: Box Filter, Gaussian Filter.

Best Filters for Different Noises¶

1. Gaussian Noise:

   • Best Filters:

     • Bilateral Filter: Effectively smooths the image while maintaining a good balance between noise removal and detail preservation, the figure showed that k=7 for bilateral filter was perfect, while k=3 kept the sharp transitions that account for noise.

     • Gaussian Filter: Provides a smooth result with minimal noise, but slightly blurs the image compared to the Bilateral Filter.

     • Adaptive Median Filter: Adapts well to varying noise levels, providing excellent noise removal and edge preservation.

   • Moderate Filters:

     • Median Filter: Effective in reducing noise but may introduce some artifacts in the image.

   • Least Effective Filters:

     • Box Filter: Results in significant blurring and loss of detail.

2. Salt and Pepper Noise:

   • Best Filters:
     • Median Filter: Eliminates all the noise while preserving the edges and details of the image.
     • Adaptive Median Filter: Eliminates most of the noise and maintains a good balance between noise removal and detail preservation.
   • Moderate Filters:
     • Bilateral Filter: Less effective in removing Salt and Pepper noise, but still provides some noise reduction.
   • Least Effective Filters:
     • Gaussian Filter: Leaves some noise in the image.
     • Box Filter: Results in significant blurring and loss of detail while failing to remove all the noise.

Summary¶

• Small Kernels: Best for preserving edges and fine details, resulting in lower MSE and higher PSNR.
• Medium Kernels: Provide a good balance between noise removal and edge preservation, with moderate MSE and PSNR.
• Large Kernels: Effective at removing significant noise but tend to blur edges and fine details, resulting in higher MSE and lower PSNR.

By understanding the effects of different kernel sizes and filter types on edge preservation, we can select the most appropriate filter for specific noise types and levels, ensuring optimal image quality and detail preservation.

Conclusion¶

In this assignment, we conducted a comprehensive analysis of various image smoothing filters, including Box filter, Gaussian filter, Median filter, Bilateral filter, and Adaptive Median filter. The key components studied in this experiment were noise removal effectiveness, edge preservation, computational efficiency, and the influence of kernel size on the performance of each filter. Quantitative metrics such as Mean Squared Error (MSE) and Peak Signal-to-Noise Ratio (PSNR) were used for comparison.

Key Findings:¶

1. Noise Removal Effectiveness:

   • Gaussian Noise: The Bilateral Filter and Gaussian Filter were the most effective in reducing Gaussian noise while preserving edges. The Adaptive Median Filter also performed well, adapting to varying noise levels.

   • Salt and Pepper Noise: The Median Filter and Adaptive Median Filter were highly effective in removing Salt and Pepper noise, preserving edges and details. The Bilateral Filter showed moderate performance.

2. Edge Preservation:

   • Small Kernel Sizes: Best for preserving edges and fine details, resulting in lower MSE and higher PSNR.

   • Medium Kernel Sizes: Provided a good balance between noise removal and edge preservation, with moderate MSE and PSNR.

   • Large Kernel Sizes: Effective at removing significant noise but tended to blur edges and fine details, resulting in higher MSE and lower PSNR.

3. Computational Efficiency:

   • Box Filter: Fastest with minimal computational cost.

   • Gaussian Filter: Very efficient with low computational cost.

   • Median Filter: Moderate computational cost, increasing with kernel size.

   • Bilateral Filter: Higher computational cost, increasing significantly with kernel size.

   • Adaptive Median Filter: High computational cost, increasing with kernel size.

   • Adaptive Mean Filter: Very high computational cost, increasing with kernel size.

1. MSE and PSNR:

   • Gaussian Noise: The Bilateral Filter and Gaussian Filter achieved the lowest MSE and highest PSNR, indicating superior performance in noise reduction and edge preservation. For the image with low details, the median filter had the best performance, which makes it the most versatile of all.

   • Salt and Pepper Noise: The Median Filter and Adaptive Median Filter achieved the lowest MSE and highest PSNR, indicating excellent noise removal and edge preservation.

Summary:¶

• Best Performance on Gaussian Noise: Bilateral Filter, Gaussian Filter, and Adaptive Median Filter.

• Best Performance on Salt and Pepper Noise: Median Filter and Adaptive Median Filter.

• Moderate Performance: Bilateral Filter on Salt and Pepper noise, Gaussian Filter on Salt and Pepper noise.

• Lower Performance: Box Filter on both Gaussian and Salt and Pepper noise, as well as adaptive mean, which had comparable performance to the box filter, but way higher computational cost, which ultimately makes it useless for most applications. As the low quality gain cannot justify the expensive computational cost.

By understanding the trade-offs between noise removal, edge preservation, computational efficiency, and the influence of kernel size, we can select the most appropriate filter for specific noise types and levels, ensuring optimal image quality and detail preservation. This analysis provides valuable insights into the strengths and weaknesses of each filter, guiding their application in various image processing tasks.